This Page Is Inserted by IFW Operations 
and is not a part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of 
the original documents submitted by the applicant. 

Defects in the images may include (but are not limited to): 

• BLACK BORDERS 

• TEXT CUT OFF AT TOP, BOTTOM OR SIDES 

• FADED TEXT 

• ILLEGIBLE TEXT 

• SKEWED/SLANTED IMAGES 

• COLORED PHOTOS 

• BLACK OR VERY BLACK AND WHITE DARK PHOTOS 

• GRAY SCALE DOCUMENTS 


IMAGES ARE BEST AVAILABLE COPY. 

As rescanning documents will not correct images, 
please do not report the images to the 
Image Problem Mailbox. 


This Page Blank (uspto) 


WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 



PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 


(51) International Patent Classification 7 : 
G06F 


A2 


(11) International Publication Number: 
(43) International Publication Date: 


WO 00733160 

8 June 2000 (08.06.00) 


(21) International Application Number: PCTAJS99/28528 

(22) International Filing Date: 2 December 1999 (02.12.99) 


(30) Priority Data: 
09/204,888 
09/205;119 
09/205,653 


3 December 1998 (03.12.98) US 
3 December 1998 (03.12.98) US 
3 December 1998 (03.12.98) US 


(71) Applicant (for all designated States except US): TELECOM 

PARTNERS LTD. [US/US]; 900 Town Center, New Britain, 
PA 18901 (US). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): ELDER1NG, Charles, A. 
[US/US); 315 Hedgerow Lane. Doylestown, PA 18901 
(US). SYLLA, M.. Lamine [SN/US]; 6 West Butler Avenue, 
New Britain, PA 18901 (US). 

(74) Agents: BLASKO, John, P. et al.; J.P. Blasko Professional 
Corp., 107 North Broad Street, Doylestown, PA 18901 (US). 


(81) Designated States: AE, AL, AM, AT, AU, AZ t BA, BB, BG, 
BR, BY, CA f CH, CN, CR, CU, CZ, DE, DK, DM, EE, 
ES, FI, GB, GD, GE, GH, GM, HR, HU, ID, IL, IN, IS, JP, 
KE, KG, KP, KR, KZ, LC. LK, LR, LS, LT, LU f LV t MD, 
- MG, MK, MN t MW, MX, NO ; NZ, PL, PT, RO, RU, SD, 
SE, SG, SI, SK, SL, TJ, TM, TO, TT, TZ, UA. UG, US, 
UZ. VN, YU, ZA, ZW, ARIPO patent (GH. GM, KE, LS, 
MW, SD, SL, SZ, TZ, UG, ZW), Eurasian patent (AM, AZ. 
BY, KG, KZ, MD, RU, TJ, TM), European patent (AT, BE, 
CH, CY, DE, DK, ES. FI, FR, GB, GR, IE, IT, LU, MC, 
NL, PT, SE), OAPI patent (BF, BJ, CF, CG, CI, CM, GA, 
GN, GW, ML, MR, NE, SN, TD, TG). 


Published 

Without international search report and to be republished 
upon receipt of that report. 


(54) Title: SUBSCRIBER CHARACTERIZATION AND ADVERTISEMENT MONITORING SYSTEM 


rl2Q 


USER 





g 



r— 

cr 

U6~> 


m 



r 130 

SOURCE 
MATERIAL 


r 140 


EPG 


132-A 'vH34 >3 



PROGRAM 
CHARACTERISTICS 
VECTORS 


SUBSCRIBER 
CHARACTERIZATION 
SYSTEM 



150 


HOUSEHOLD 
DEMOGRAPHIC 
CHARACTERISTICS 
{AVE. AND SESSION) 


MflO 


PROGRAM 
DEMOGRAPHIC 
VECTORS 


(57) Abstract 


A subscriber characterization and advertisement monitoring system (100) is presented in which subscriber viewing habits are monitored 
to determine demographic profiles. These profiles can be utilized for the matching of advertisements to subscribers based on their viewing 
habits and estimated demographics and product interests. The system (100) can be run locally in a television set-top (1808) or can be 
run in client server mode where channel selections are transmitted from the residence (1800) to a centralized switching location (server) 
(1840) such as a telephone office or Internet Service Provider. In client-server mode the channel selections are monitored at the centralized 
location (1840) which also performs the subscriber characterization. The system also provides the ability to monitor if advertisements were 
viewed and for what duration. 
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TITLE 

Subscriber characterization and 
advertisement monitoring system 

5 

Background of the Invention 

Cable television service providers have typical I v 
provided one-way broadcast services but now offer high-spc--.* : 
data services and can combine traditional analog broadcast ; 

10 with digital broadcasts and access to Internet web siz^r . 
Telephone companies can offer digital data and via-- 
programming on a switched basis over digital subscriber };:;•- 
technology. Although the subscriber may only be present.-: 
with one channel at a time, channel change requests 

15 instantaneously transmitted to centralized switch::.: 
equipment and the subscriber can access the programming ii. 
broadcast-like manner. Internet Service Providers 
offer Internet access and can offer access to text, audi., 
and video programming which can also be delivered i: : ..: 

2 0 broadcast -like manner in which the subscriber seh-*-:.: 
"channels" containing programming of interest. l^u.*. 
channels may be offered as part of a video programm : : 
service or within a data service and can be presented wir I.:;, 
an Internet browser. 

25 Advertisements are a part of daily life and certainly 

an important part of entertainment programming, where t re- 
payments for advertisements cover the cost of netwr : •: 
television. A method, which provides a flexible billing i I ir. 
to cable network users based on the amount of advertiser, 

30 viewed is described in U.S. Patent No. 5,532,735, v;li:;.\ 
discloses a method of advertisement selection :\: 
interactive services. A user associated with an interact 3 v- 
TV is presented with a program and a set of advertisement/. 
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The user can. indicate the amount of advertisements in the 
set of advertisements he wants to view. 

While advertisements are sometimes beneficial to 
subscribers and deliver desired information regarding 
specific products or services, consumers generally view 
advertising as a "necessary evil" for broadcast - type 
entertainment. For example, a method for obtaining 
information on advertised services or products is described 
in U.S. Patent No. 5,708', 478, which discloses a computer 
system for enabling radio listeners and television watchers 
to obtain advertising information. The system includes steps 
of determining whether an incoming video or audio signal 
includes advertisement specific data of an advertiser and 
capturing and storing the advertiser specific data. 

Manufacturers pay an extremely high price to present, 
in 3 0 seconds or less , an advertisement for their product , 
which they hope a consumer will watch. Unfortunately for the 
manufacturer, the consumer frequently uses that interval of 
time to check the programming being presented on the other 
channels, and may not watch any of the advertisement. 
Alternately, the consumer may mute the channel and ignore 
what the manufacturer has presented. In any case the 
probability that the consumer has watched the advertisement 
is quite low. It is not until millions of dollars have been 
spent on an advertising campaign that a manufacturer can 
determine that the ads have been effective. This is 

presently accomplished by monitoring sales of the product or 
TV programs or channels viewed by users as disclosed in 
various public documents. As an example, U.S. Patent No. 
4,546,382 discloses a television and market research data 
collection system and method. A data collection unit 
containing a memory stores data as to which of the plurality 
of TV modes are in use, which TV channel is being viewed as 
well as input from a suitable optical scanning device for 
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collecting information about user's product purchases. 
Another system described in U.S. Patent No. 4,258,385 
discloses a television audience measuring system. The system 
monitors and stores information representative of channel 
5 identification, the time at which the channel is selected 
and the time at which the selection of a channel is 
terminated. U.S. Patent No. 5,608,445 discloses also a 
method and device for data capture in television viewer 
research. Devices are attached to a video installation in 

10 order to determine to which channel a set is tuned. 

With the advent of the Internet manufacturers and 
service providers have found ways to selectively insert 
their advertisements based on a subscribers requests for 
information. As an example, an individual who searches for 

15 "cars" on the Internet may see an advertisement for a 
particular type of car. Various internet -based advertising 
use this method. The product literature from IMGIS Inc.," Ad 
Force," printed. from the World Wide Web site 
http://www.starpt.com/core/ad__Target.html on June 30, 1993 

20 discloses, an ad targeting system. The system delivers ads to 
web sites visitors based on the content of the web page, 
time of day, day of the week, keyword, by the number of 
times a visitor sees an advertisement and by the order in 
which a series of advertisements are shown to a visitor. 

25 Nevertheless, unless the subscriber actually goes to the 
advertised web site, there is no way to determine if the 
advertisement has been watched. As the content on the 
Internet migrates to multimedia programming including audio 
and video, the costs for the advertising will increase, but 

30 unless the advertiser can be sure that a significant 
percentage of the message was watched or observed, the 
advertising is ineffective. Prior art products for 
generating reports of ad campaign are generally PC-centric 
as described in various product literature which include the 
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product literature from Doubleclick Inc., "Doubleclick: 
Reporting," printed from the World Wide Web (WWW) sir- 
http : / /www. doubleclick . net /dart /howi_repo . htm on June 1 , 
1998, which discloses the reporting capabilities cf 
5 Doubleclick's Dynamic Advertising Reporting & Target: inn 
(DART) product. The information in the reports includes 
daily impressions by advertisement type, average impressioi: 
per day of week and by hour of day. The average resccn^e 
rate per user is also included in the reports. The prcc;ir: 
10 literature from Netgravity Inc. "AdServer 3," printed from 
the World Wide Web site http : //www. netgravity . com/product;:/ 
on July 9, 1998 discloses Netgravity' s x Adserver 3 product 
for online advertisement. The product generates report.;; 
including the profiles of visitors who viewed an ad and ^ : r e 
15 traffic throughout the day, week, month and year. 

The product literature from Media Metrix "Frequent I v 
Asked Questions" , printed from the World Wide Web - : to 
http: / /www. mediametrix. com/ interact_mmfaq.htm on June 2(\ , 
1998 discloses Media Metrix software, PC Meter, that runs- in 
the background of a PC and monitors everything being dor.-.- en 
that machine . It determines who is using the PC by aao , 
income, gender and geographic region and tracks usage oi 
software application, commercial online services an;: 
detailed page level viewing of the World Wide Web. re- 
marketing literature from Matchlogic Inc . , "Centralized A1 
Management , " printed from the World Wide Web s ; ie 
http://www.matchlogic.com/docs/services2.htm on July 1, 
discloses Matchlogic services for ad management . Tl:<v 
services include delivering advertisements based on :: i 
30 defined targeting criteria, generating reports on how -i.-rv. 
unique viewers saw which banner and how many times it w.j.; 
viewed. The product literature from Accipiter Inc.." 
Accipiter AdManager 2.0," printed from the World Wide wVr 
site http : / /www. accipiter . com/product s/ADManager/f ab .htm J c:. 


20 


25 
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July 9, 1998 discloses Accipiter's ad management system. 
After delivering an advertisement based on pre-defined 
criteria, the system can generate reports on an ad campaign. 
The reports include visitors' demographic data, number of 
5 impressions and clicks generated from the entire site and by 
each ad and advertiser. 

In order to deliver more targeted programming and 
advertising to subscribers, it is necessary to understand 
their likes and dislikes to a greater extent than is 

10 presently done today. Systems which identify subscriber 
preferences based on their purchases and responses to 
questionnaires allow for the targeted marketing of 
literature in the mail, but do not in any sense allow for 
the rapid and precise delivery of programming and 

15 advertising which is known to have a high probability of 
acceptance to the subscriber. Other systems give users the 
possibility to chose their programming as described in U.S. 
Patent No. 5,223,924 which discloses a system and method for 
automatically correlating user preferences with a TV program 

2 0 information database. The system includes a processor that 
performs "free text" search techniques to correlate the 
downloaded TV program information with the viewer's 
preferences. This system requires an interaction between the 
users and the programming. The white paper from Net 

25 Perceptions corporation entitled %x Adding Value in the 
Digital Age" and printed from the World Wide Web site 
ht tp : //www . netperceptions . com/product s/ white -papers . html on 
June 30, 1998 discloses how the GroupLens Recommendation 
Engine gives online businesses the ability to target and 

30 personalize services, content, products and advertising. A 
learning process learns personal information about an 
individual using explicit and implicit ratings, a prediction 
process predicts user preference using collaborative 
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filtering and the recommendation process recommends products 
or services to users based on predictions. 

The product literature from Aptex software Inc., 
"SelectCast for Commerce Servers," printed from the World 
Wide Web site http://www.aptex.com/products-selectcast- 
commerce.htm on June 30, 1998 describes the product 
SelectCast for Commerce Servers. it personalizes online 
shopping based on observed user behavior. User interests are 
learned based on the content they browse, the promotions 
they click and the products they purchase. 

In order to determine which programming or advertising 
is appropriate for the subscriber, 'knowledge of that 
subscriber and the subscriber product and programming 
preferences is required. Different methods are being used to 
15 gain knowledge of user's preferences and to profile the 
users. Generally, these methods use content or data mining 
technologies to profile users or predict their preferences. 
Another technique for predicting user's preferences is based 
on the use of collaborative filtering as described in U.S. 
20 Patent No. 5,704,017 which discloses a collaborative 
filtering system utilizing a belief network. The system 
learns a belief network using prior knowledge obtained from 
an expert in a given field of decision making and a database 
containing empirical data such as users' attributes as well 
25 as their preferences in that decision making field. The 
belief network can determine the probability of the unknown 
preferences of the user given the known attributes and thus 
predicts the preference most likely to be desired by the 


30 


user 


The product literature from Aptex software Inc., 
"SelectCast for Ad Servers," printed from the World Wide Web 
site http://www.aptex.com/products-selectcast-ads.htm on 
June 30, 1998 discloses an ad targeting system from Aptex 
Software Inc. The system employs neural networks and a 
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context vector data model to optimize relationships between 
users and content. It provides user profiling by mining the 
context and content of all actions including clicks, 
queries, page views and ad impressions. Aptex's technology 
5 uses a context vector data modeling technique described in 
U.S. Patent No. 5,619,709 which discloses a system and 
method of context vector generation and retrieval. Context 
vectors represent conceptual relationships among information 
items by quantitative means. A neural network operates on a 

10 training corpus of records to develop relationship-based 
context vectors based on word proximity and co- importance . 
Geometric relationships among context vectors are 
representative of conceptual relationships among their 
associated items. 

15 The product Data sheet from Open Sesame, "Learn 

Sesame," printed from the World Wide Web site 
http://www.opensesame.com/prod_04.html on July 09, 1998 
discloses Open Sesame's personalization product for Web 
enterprises. It learns about users automatically from their 

20 browsing behavior. 

The product literature from Engage Technologies, 
"Engage . Discover , " printed from the World Wide Web site 
http://www.engagetech.com on July 09, 1998 discloses Engage 
Technologies' product for user profiling. User-disclosed 

25 information such as interest, demographics and opinions are 
combined with anonymous clickstream data that describes 
where users come from before visiting the site, how long 
they stay, and what pages or types of pages they visit most 
frequently to build the visitor profile. 

30 The marketing literature from Broadvision, "The Power 

of Personalization" , printed from the World Wide Web site 
http : //www . broadvision . com/content /corporate/brochure/Broch4 
.htm on August 21, 1998 discloses Broadvision One-to-One 
application profiling system. The system learns about users 
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through a variety of techniques including registration, 
questionnaires, observation and integration of historical 
and externally generated data. 

The marketing literature from Firefly Corporation, 
"Firefly passport Office," printed from the World Wide Web 
site http://www.firefly.net/company/PassportOffice.html on 
June 20, 1998 discloses Firefly's Relationship Management 
software. The software enables online businesses to create, 
extend and manage personal profiles for every user. 

Specific information regarding a subscriber's viewing 
habits or the Internet web sites they have accessed can be 
stored for analysis, but such records ar*e considered private 
and subscribers are not generally willing to have such 
information leave their control. Although there are 
regulatory models, which permit the collection of such data 
on a "notice and consent" basis, there is a general tendency 
towards legal rules, which prohibit such raw data to be 
collected. 

With the migration of services from a broadcast based 
model to a client-server based model in which subscribers 
make individualized request for programming to an Internet 
access provider or content provider, there is opportunity to 
monitor the subscriber viewing characteristics to better 
provide them with programming and advertising which will be 
of interest to them. A server may act as a proxy for the 
subscriber requests and thus be able to monitor what a 
subscriber has requested and is viewing. Since subscribers 
may not want this raw data to be utilized, there is a need 
for a system which can process this information and generate 
statistically relevant subscriber profiles. These profiles 
should be accessible to ethers on the network who may wish 
to determine if their programming or advertisements are 
suitable for the subscriber. In a broadcast -based model, the 
information to be processed can be embedded within the TV 
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program or broadcast separately and can be in form of an 
electronic program guide (EPG) or text information related 
to the program. As an example, U.S. Patent No. 5,579,055 
discloses an electronic program guide (EPG) and text channel 
5 data controller. The text and EPG data are embedded in the 
vertical blanking interval of the .video signal and 
extracted, at reception, by the data controller. The EPG 
contains information fields such as program category, 
program subcategory and program content description. U.S. 

10 Patent No. 5,596,373 discloses also a method and apparatus 
for providing program oriented information in a multiple 
station broadcasting system. The EPG Hata includes guide 
data, channel data and program data. The program data 
includes among other information, the program title, the 

15 program category, the program sub-category and a detailed 
description of the program. 

For the foregoing reasons, there is a need for an 
advertisement monitoring system which can monitor which 
advertisements have been viewed by a subscriber. There is 

20 also a need for a subscriber characterization system which 
can generate and store subscriber characteristics which 
reflect the probable demographics and preferences of the 
subscriber and household. 


25 

Summary Of The Invention 

The present invention encompasses a system for 
determining to what extent an advertisement has been viewed 
by a subscriber or household. 
30 In a preferred embodiment subscriber selection data 

including the channel selected and the time at which it was 
selected are recorded. Advertisement related information 
including the type of product, brand name, and other 
descriptive information which categorizes the advertisement 
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on 


is extracted from the advertisement or text informs ti 
related to the advertisement including clo S ed captioning 
text.. Based on the subscriber selection data a record of 
what percentage of the advertisement was watched is created. 
This record can subsequently be used to make a measure of 
the effectiveness of the advertisement. 

In a preferred embodiment the text information related 
to the advertisement is processed using context mining 
techniques which allow for classification of the 
advertisement and extraction of key data including product 
type and brand. Context mining .techniques allow for 
determination of a product type, product brand name and in 
the case of a product which is not sold with a particular 
brand name, a generic name for the product. 

The present invention can also be realized in a client - 
server mode in which case the subscriber executes channel 
changes at the client side of the network which are 
transmitted to the server side and fulfilled by the routing 
of a channel to the subscriber. The server side monitors 
the subscriber activity and stores the record of channel 
change requests. Advertisement related information is 

retrieved from the server side, which contains the 
advertising material itself, retrieves the advertising 
material from a third party, or analyzes the data stream 
carrying the advertising to the subscriber. The server side 
extracts descriptive fields from the advertisement and based 
on the subscriber selection data, determines the extent to 
which the advertisement was viewed by the subscriber. As an 
example the system can determine the percentage of the 
advertisement that was viewed by the subscriber. 

The present invention includes a system for 
characterizing subscribers watching video or multimedia 
programming based on monitoring their detailed selection 
choices including the time duration of their viewing, the 

10 
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volume the programming is listened at, the program 
selection, and collecting text information about that 
programming to determine what type cf programming the 
subscriber is most interested in. In addition, the system 
5 can generate a demographic description of the subscriber cr 
household which describes the probable age, income, gender 
and other demographics. The resulting characterization 
includes probabilistic determinations of what other 
programming or products the subscriber/household will be 

10 interested in. 

In a preferred embodiment, the textual information 
which describes the programming is obtained by context 
mining of text associated with the programming. The 
associated text can be from the closed- captioning data 

15 associated with the programming, an electronic program 
guide, or from text files associated with or part of the 
programming itself . 

The system can provide both session measurements which 
correspond to a profile obtained over a viewing session, or 

20 an average profile which corresponds to data obtained over 
multiple viewing sessions. 

The present invention also encompasses the use of 
heuristic rules in logical form or expressed as conditional 
probabilities to aid in forming a subscriber profile. The 

25 heuristic rules in logical form allow the system to apply 
generalizations which have been . learned from external 
studies to obtain a characterization of the subscriber. In 
the case of conditional probabilities, determinations of the 
probable content of a program can be applied in a 

30 mathematical step to a matrix of conditional probabilities 
to obtain probabilistic subscriber profiles indicating 
program and product likes and dislikes as well for 
determining probabilistic demographic data. 

One advantage of the present invention is that it 

11 
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allows consumers the possibility to permit access to 
probabilistic information regarding their household 
demographics and programming/product preferences, without 
revealing their specific viewing history. Subscribers may 
elect to permit access to this .information in order to 
receive advertising which is more targeted to their 
likes/dislikes. Similarly, a subscriber may wish to sell 
access to this statistical data in order to receive revenue 
or receive a discount on a product or a service. 

Another advantage of the present invention is that the 
resulting probabilistic information can be stored locally 
and controlled by the subscriber, or can' be transferred to a 
third party which can provide access to the subscriber 
characterization. The information can also be encrypted to 
15 prevent unauthorized access in which case only the 
subscriber or someone authorized by the subscriber can 
access the data. 

The present invention includes also a system for 
characterizing subscribers watching, video or multimedia 
2 0 programming based on monitoring the requests made by the 
subscriber for programming to a server which contains the 
content or which requests the content from a third party. 
The server side of the network is able to monitor the 
subscriber's detailed selection choices including the time 
25 duration of their viewing, the volume the programming is 
listened at, and the program selection. 

The server side collects text information about that 
programming to determine what type of programming the 
subscriber is most interested in. In addition the system 
can generate a demographic description of the subscriber or 
household which describes the probable age, income, gender 
and other demographics. The resulting characterization 
includes probabilistic determinations of what other 


30 
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programming or products the subscriber/household will be 
interested in. 

These and other features and objects of the invention 
will be more fully understood from the following detailed 
5 description of the preferred embodiments which should be 
read in light of the accompanying drawings. 

Brief Description of the Drawings 

The accompanying drawings, which are incorporated in 
10 and form a part of the specification, illustrate the 
embodiments of the present invention and, together with the 
description serve to explain the ^principles of the 
invention. 

In the drawings: 

15 FIG. 1 shows a context diagram for a subscriber 

character! zat ion system . 

FIG. 2 illustrates a block diagram for a realization of 
a subscriber monitoring system for receiving video signals ; 

FIG. 3 illustrates a block diagram of a channel 
2 0 processor; 

FIG. 4 illustrates a block diagram of a computer for a 
realization of the subscriber monitoring system; 

FIG. 5 illustrates a channel sequence and volume over a 
twenty- four (24) hour period; 
25 FIG. 6 illustrates a time of day detailed record; 

FIG. 7 illustrates a household viewing habits 
scatistical table; 

FIG. 8A illustrates an entity-relationship diagram for 
the generation of program characteristics vectors; 
30 FIG. 8B illustrates a flowchart for program 

characteri zat ion ; 

FIGS. 9A illustrates a deterministic program category 
vector ; 

13 
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FIG. 9B illustrates a deterministic program sub- 
category vector ; 

FIG. 9C illustrates a deterministic program rating 
vector ; 

5 FIG. 9D illustrates a. probabilistic program category 

vector ; 

FIG. 9E illustrates a probabilistic program sub- 
category vector; 

FIG. 9F illustrates a probabilistic program content 
10 vector; 

FIG. 10A illustrates a set of logical heuristic rules; 
FIG. 10B illustrates a set of heuristic rules expressed 
in terms of conditional probabilities ;. 

FIG. 11 illustrates an entity-relationship diagram for 
15 the generation of program demographic vectors ; 

FIG. 12 illustrates a program demographic vector ; 
FIG. 13 illustrates an entity-relationship diagram for 
the generation of household session demographic data and 
household session interest profiles ; 
20 FIG * 14 illustrates an entity- relationship diagram for 

the generation of average and session household demographic 
characteristics ; 

FIG. 15 illustrates average and session household 
demographic data ; 

25 FIG - 16 illustrates an entity-relationship diagram for 

generation of a household interest profile; 

FIG. 17 illustrates household interest profile 

including programming and product profiles; 

FIG. 18 illustrates a client-server architecture for 

30 realizing the present invention; and 

FIG. 19 illustrates an advertisement monitoring table. 
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Detailed Description 
Of The Preferred Embodiment 

In describing a preferred embodiment of the invention 
illustrated in the drawings, specific terminology will be 
5 used for the sake of claricy. However, the invention is not 
intended to be limited to the specific terms so selected, 
and it is to be understood that each specific term includes 
all technical equivalents which operate in a similar manner 
to accomplish a similar purpose. 

10 With reference to the drawings, in general, and FIGS. 1 

through 19 in particular, the apparatus of the present 
invention is disclosed. 

The present invention is directed at an apparatus for 
monitoring which advertisements are watched by a subscriber 

15 or a household. 

In the present system the programming viewed by 
the subscriber, both entertainment and advertisement, can be 
studied and processed by the subscriber characterization 
system to determine the program characteristics. This 

20 determination of the program characteristics is referred to 
as a program characteristics vector. The vector may be a 
truly one-dimensional vector, but can also be represented as 
an n dimensional matrix which can be decomposed into 
vectors. For advertisements, the program characteristics 

2 5 vector can contain information regarding the advertisement 
including product type, features, brand or generic name, or 
other relevant advertising information. 

The subscriber profile vector represents a profile of 
the subscriber (or the household of subscribers) and can be 

30 in the form of a demographic profile (average or session) or 
a program or product preference vector. The program and 
product preference vectors are considered to be part of a 
household interest profile which can be thought of as an n 
dimensional matrix representing probabilistic measurements 
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of subscriber interests. 

In the case that the subscriber profile vector , 
demographic profile, the subscriber profile vector indict:- 
a probabilistic measure of the age of the subscriber - 
average age of the viewers in the household, sex of -i.. 
subscriber/ income range of the subscriber or household, 
other such demographic data. Such information compr i 
household demographic characteristics and is composed : : 
both average and session values. Extracting a single se- .! 
values from the household demographic characteristics ■ ,- : 
correspond to a subscriber profile vector. 

The household interest profile - can contain t 
programming and product profiles, with programming pro:.;,.- 
corresponding to probabilistic determinations* of 
programming the subscriber (household) is likely tr. \- 
interested in, and product profiles corresponding to v.n- 
products the subscriber (household) is likely \ u c . i-. 
interested in. These profiles contain both an average v.; ; ... 
and a session value, the average value being a time avc-r .-k- 
of data, where the averaging period may be several d.w:, 
weeks, months, or the time between resets of unit. 

Since a viewing session is likely to be dominated :y : 
particular viewer, the session values may, in ■■ 
circumstances, correspond most closely to the subscni ■■! 
values, while the average values may, in some circurnst an -. . 
correspond most closely to the household values. 

FIG . 1 depicts the context diagram of a pretv: : • : 
embodiment of a Subscriber Characterization System ■■ / \ 
100. A context diagram, in combination with em !••-■ 
relationship diagrams , provide a basis from which 
skilled in the art can realize the present invention . '1 i. 
present invention can be realized in a number of prograr.::;.r. i 
languages including C, C++, Perl, and Java, although 
scope of the invention is not limited by the choice cl. /> 
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particular programming language or tool. Object oriented 
languages have several advantages in terms of construction 
of the software used to realize the present invention, 
although the present invention can be realized in procedural 
5 or other types of programming languages known to those 
skilled in the art. 

In generating a subscriber profile, the SCS 100 
receives from a user 120 commands in the form of a volume 
control signal 124 or program selection data 122 which can 

10 be in the form of a channel change but may also be an 
address request which requests the delivery of programming 
from a network address. A record signal" 126 indicates that 
the programming or the address of the programming is being 
recorded by the user. The record signal 126 can also be a 

15 printing command, a tape recording command, a bookmark 
command or any other command intended to store the program 
being viewed, or program address, for later use. 

The material being viewed by the user 120 is referred 
to as source material 130. The source material 130, as 

20 defined herein, is the content that a subscriber selects and 
may consist of analog video, Motion Picture Expert Group 
(MPEG) digital video source material, other digital or 
analog material , Hypertext Markup Language (HTML) or other 
type of multimedia source material . The subscriber 

25 characterization system 100 can access the source material 
130 received by the user 120 using a start signal 132 and a 
stop signal 134, which control the transfer of source 
related text 136 which can be analyzed as described herein. 

In a preferred embodiment, the source related text 136 

30 can be extracted from the source material 130 and stored in 
memory. The source related text 136, as defined herein, 
includes source related textual information including 
descriptive fields which are related to the source material 
130, or text which is part of the source material 130 
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itself. The source related text 13 6 can be derived from a 
number of sources including but not limited to closed 
captioning information, Electronic Program Guide (SPG) 
material, and text information in the source itself (e.g. 
text in HTML files) . 

Electronic Program Guide (EPG) 140 contains information 
related to the source material 130 which is useful to the 
user 120. The EPG 140 is typically a navigational tool 
which contains source related information including but not 
limited to the programming category, program description, 
rating, actors, and duration. The structure and content of 
EPG data is described in detail in US Patent 5,596,373 
assigned to Sony Corporation and Sony Electronics which is 
herein incorporated by reference'. As shown in FIG. 1, the 
EPG 14 0 can be accessed by the SCS 10 0 by a request EPG data 
signal 142 which results in the return of a category 144, a 
sub-category 14 6, and a program description 14 8. EPG 
information can potentially include fields related to 
advertising . 

In one embodiment of the present invention, EPG data is 
accessed and program information such as the category 144, 
the sub-category 146, and the program description 148 are 
stored in memory. 

In another embodiment of the present invention, the 
source related text 136 is the closed captioning text 
embedded in the analog or digital video signal. Such closed 
captioning text can be stored in memory for processing to 
extract the program characteristic vectors 150. 

One of the functions of the SCS 100 is to generate the 
program characteristics vectors 150 which are comprised of 
program characteristics data 152, as illustrated in FIG. 1. 
The program characteristics data 152, which can be used to 
create the program characteristics vectors 150 both in 
vector and table form, are examples of source related 
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information which represent characteristics of the source 
material. In a preferred embodiment, the program 
characteristics vectors 150 are lists of values which 
characterize the programming (source) material in according 
5 to the category 144, the sub-category 146, and the program 
description 148. The present invention may also be applied 
to advertisements, in which case program characteristics 
vectors contain, as an example, a product category, a 
product sub-category, and a brand name. 

10 As illustrated in FIG. 1, the SCS 100 uses heuristic 

rules 160. The heuristic rules 160, as described herein, 
are composed of both logical heuristic rules as well as 
heuristic rules expressed in terms of conditional 
probabilities. The heuristic rules 160 can be accessed by 

15 the SCS 100 via a request rules signal 162 which results in 
the transfer of a copy of rules 164 to the SCS 100. 

The SCS 100 forms program demographic vectors 170 from 
program demographics 172, as illustrated in FIG. 1. The 
program demographic vectors 170 also represent 

20 characteristics of source related information in the form of 
the intended or expected demographics of the audience for 
which the source material is intended. 

Subscriber selection data 110 is obtained from the 
monitored activities of the user and in a preferred 

25 embodiment can be stored in a dedicated memory. In an 
alternate embodiment, the subscriber selection data 110 is 
stored in a storage disk. Information which is utilized to 
form the subscriber selection data 110 includes time 112, 
which corresponds to the time of an event, channel ID 114, 

30 program ID 116, volume level 118, channel change record 119, 
and program title 117. A detailed record of selection data 
is illustrated in FIG. 6. 

In a preferred embodiment, a household viewing habits 
195 illustrated in FIG. 1 is computed from the subscriber 
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selection data 110. The SCS 100 transfers household viewing 
data 197 to form household viewing habits 195. The 
household viewing data 197 is derived from the subscriber 
selection data 110 by looking at viewing habits at a 
particular time of day over an extended period of time, 
usually several days or weeks, and making some 
generalizations regarding the viewing habits during that 
time period. 

The program characteristics vector 150 is derived from 
the source related text 13 6 and/or from the EPG 14 0 by 
applying information retrieval techniques. The details of 
this process are discussed in accordance with FIG. 8. 

The program characteristics vector 150 is used in 
combination with a set of the heuristic rules 160 to define 
a set of the program demographic vectors 170 illustrated in 
FIG. 1 describing the audience the program is intended for. 

One output of the SCS 100 is a household profile 
including household demographic characteristics 190 and a 
household interest profile 180. The household demographic 
characteristics 190 resulting from the transfer of household 
demographic data 192, and the household interest profile 
180, resulting from the transfer of household interests data 
182. Both the household demographics characteristics 190 
and the household interest profile 180 have a session value 
25 and an average value, as will be discussed herein. 

The monitoring system depicted in FIG. 2 is responsible 
for monitoring the subscriber activities, and can be used to 
realize the SCS 100. In a preferred embodiment, the 

monitoring system of FIG. 2 is located in a television set- 
30 top device or in the television itself. in an alternate 
embodiment, the monitoring system is part of a computer 
which receives programming from a network. 

In an application of the system for television 
services, an input connector 220 accepts the video signal 

20 
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coming either from an antenna, cable television inout, or 
other network. The video signal can be analog or Digital 
MPEG. Alternatively, the video source may be a video stream 
or other multimedia stream from a communications network 
5 including the Internet. 

In the case of either analog or digital video, selected 
fields are defined to carry EPG data cr closed captioning 
text. For analog video, the closed captioning text is 
embedded in the vertical blanking interval (VBI) . As 

10 described in US Patent 5,579,005, assigned to Scientific- 
Atlanta, Inc., the EPG information can be carried in a 
dedicated channel or embedded in the X VBI. For digital 
video, the closed captioning text is carried as video user 
bits in a user__data field. The EPG data is transmitted as 

15 ancillary data and is multiplexed at the transport layer 
with the audio and video data. 

Referring to FIG. 2, a system control unit 200 receives 
commands from the user 120, decodes the command and forwards 
the command to the destined module. In a preferred 

20 embodiment, the commands are entered via a remote control to 
a remote receiver 205 or a set of selection buttons 2 07 
available at the front panel of the system control unit 200. 
In an alternate embodiment, the commands are entered by the 
user 12 0 via a keyboard. 

25 The system control unit 200 also contains a Central 

Processing Unit (CPU) 203 for processing and supervising all 
of the operations of the system control unit 2 00, a Read 
Only Memory (ROM) 202 containing the software and fixed 
data, a Random Access Memory (RAM) 204 for storing data. CPU 

30 203, RAM 204, ROM 202, and I/O controller 201 are attached 
to a master bus 206. A power supply in a form of battery 
can also be included in the system control unit 200 for 
backup in case of power outage . 

An input/output (I/O) controller 201 interfaces the 
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system control unit 2 00 with external devices. In a 
preferred embodiment, the I/O controller 201 interfaces to 
the remote receiver 205 and a selection button such as the 
channel change button on a remote control. In an alternate 
5 embodiment, it can accept input from a keyboard or a mouse. 

The program selection data 122 is forwarded to a 
channel processor 210. The channel processor 210 tunes to a 
selected channel and the media stream is decomposed into its 
basic components: the video stream, the audio stream, and 
10 the data stream. The video stream is directed to a video 
processor module 230 where it is decoded and further 
processed for display to the TV screen. The audio stream is 
directed to an audio processor 24 0 for decoding and output 
to the speakers. 

15 The data stream can be EPG data, closed captioning 

text, Extended Data Service (EDS) information, a combination 
of these, or an alternate type of data. In the case of EDS 
the call sign, program name and other useful data are 
provided. In a preferred embodiment, the data stream is 
20 stored in a reserved location of the RAM 204. In an 
alternate embodiment, a magnetic disk is used for data 
storage. The system control unit 200 writes also in a 
dedicated memory, which in a preferred embodiment is the RAM 
2 04, the selected channel, the time 112 of selection, the 
2 5 volume level 118 and the program ID 116 and the program 
title 117. Upon receiving the program selection data 122, 
the new selected channel is directed to the channel 
processor 210 and the system control unit 200 writes to the 
dedicated memory the channel selection end time and the 
program title 117 at the time 112 of channel change. The 
system control unit 200 keeps track of the number of channel 
changes occurring during the viewing time via the channel 
change record 119. This data forms part of the subscriber 
selection data 110. 
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The volume control signal 124 is sent to the audio 
processor 240. In a preferred embodiment, the volume level 
118 selected by the user 120 corresponds to the listening 
volume. In an alternate embodiment, the volume level 118 
5 selected by the user 120 represents a volume level to 
another piece of equipment such as an audio system (home 
theatre system) or to the television itself. In such a 
case, the volume can be measured directly by a microphone or 
other audio sensing device which can monitor the volume at 

10 which the selected source material is being listened. 

A program change occurring while watching a selected 
channel is also logged by the system control unit 2C0. 
Monitoring the content of the program at the time of the 
program change can be done by reading the content of the 

15 EDS. The EDS contains information such as program title, 
which is transmitted via the VBI . A change on the program 
title field is detected by the monitoring system and logged 
as an event. In an alternate embodiment, an EPG is present 
and program information can be extracted from the EPG. In a 

20 preferred embodiment, the programming data received from the 
EDS or EPG permits distinguishing between entertainment 
programming and advertisements. 

FIG. 3 shows the block diagram of the channel processor 
210. In a preferred embodiment, the input connector 220 

25 connects to a tuner 300 which tunes to the selected channel. 
A local oscillator can be used to heterodyne the signal to 
the IF signal. A demodulator 302 demodulates the received 
signal and the output is fed to an FEC decoder' 304. The data 
stream received from the FEC decoder 304 is, in a preferred 

3 0 embodiment, in an MPEG format. In a preferred embodiment, 
system demultiplexer 306 separates out video and audio 
information for subsequent decompression and processing, as 
well as ancillary data which can contain program related 
information . 
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The data stream presented to the system demultiplexer 
306 consists of packets of data including video, audio and 
ancillary data. The system demultiplexer 306 identifies each 
packet from the stream ID and directs the stream to the 
corresponding processor. The video data is directed to the 
video processor module 230 and the audio data is directed to 
the audio processor 240. The ancillary data can contain 
closed captioning text, emergency messages, program guide, 
or other useful information. 

Closed captioning text is considered to be ancillary 
data and is thus contained in the video stream. The system 
demultiplexer 306 accesses the user' data field of the video 
stream to extract the closed captioning text. The program 
guide, if present, is carried on data stream identified by a 
specific transport program identifier. 

In an alternate embodiment, analog video can be used. 
For analog programming, ancillary data such as closed 
captioning text or EDS data are carried in a vertical 
blanking interval . 

FIG. 4 shows the block diagram of a computer system for 
a realization of the subscriber monitoring system based on 
the reception of multimedia signals from a bi-directional 
network. A system bus 422 transports data amongst the CPU 
2 03, the RAM 2 04, Read Only Memory - Basic Input Output 
System (ROM-BIOS) 4 06 and other components. The CPU 203 
accesses a hard drive 400 through a disk controller 402. The 
standard input/output devices are connected to the system 
bus 422 through the I/O controller 201. A keyboard is 
attached to the I/O controller 201 through a keyboard port 
416 and the monitor is connected through a monitor port 418. 
The serial port device uses a serial port 420 to communicate 
with the I/O controller 201. Industry Standard Architecture 
(ISA) expansion slots 4 08 and Peripheral Component 
Interconnect (PCI) expansion slots 410 allow additional 
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cards to be placed into the computer. In a preferred 
embodiment, a network card is available to interface a local 
area, wide area, or other network. 

FIG. 5 illustrates a channel sequence and volume over a 
5 twenty- four (24) hour period. The Y-axis represents the 
status of the receiver in terms of on/off status and volume 
level. The X-axis represents the time of day. The channels 
viewed are represented by the windows 501-506, with a first 
channel 502 being watched followed by the viewing of a 

10 second channel 504, and a third channel 506 in the morning. 
In the evening a fourth channel 501 is watched, a fifth 
channel 503, and a sixth channel 505. A channel change is 
illustrated by a momentary transition to the u off" status 
and a volume change is represented by a change of level on 

15 the Y-axis. 

A detailed record of the subscriber selection data 110 
is illustrated in FIG. 6 in a table format. A time column 
602 contains the starting time of every event occurring 
during the viewing time. A Channel ID column 604 lists the 

20 channels viewed or visited during that period. A program 
title column 603 contains the titles of all programs viewed. 
A volume column 601 contains the volume level 118 at the 
time 112 of viewing a selected channel. 

A representative statistical record corresponding to 

25 the household viewing habits 195 is illustrated in FIG. 7. 
In a preferred embodiment, a time of day column 700 is 
organized in period of time including morning, mid-day, 
afternoon, night, and late night. In an alternate 

embodiment, smaller time periods are used. A minutes 

30 watched column 702 lists, for each period of time, the time 
in minutes in which the SCS 100 recorded delivery of 
programming. The number of channel changes during that 
period and the average volume are also included in that 
table in a channel changes column 704 and an average volume 
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column 706 respectively. The last row of the statistical 
record contains the totals for the items listed in the 
minutes watched column 702, the channel changes column 704 
and the average volume 706. 

FIG. 8A illustrates an entity-relationship diagram for 
the generation of the program characteristics vector 150. 
The context vector generation and retrieval technique 
described in US Patent 5,619,709, which is incorporated 
herein by reference, can be applied for the generation of 
the program characteristics vectors 150. Other techniques 
are well known by those skilled in the art. 

Referring to FIG. 8A, the source "material 13 0 or the 
EPG 14 0 are passed through a program characterization 
process 800 to generate the program characteristics vectors 
150. The program characterization process 800 is described 
in accordance with FIG. SB. Program content descriptors 
including a first program content descriptor 802, a second 
program content descriptor 8 04 and an nth program content 
descriptor 806, each classified in terms of the category 
144, the sub-category 146, and other divisions as identified 
in the industry accepted program classification system, are 
presented to a context vector generator 820. As an example, 
the program content descriptor can be text representative of 
the expected content of material found in the particular 
program category 144. In this example, the program content 
descriptors 802, 804 and 806 would contain text 
representative of what would be found in programs in the 
news, fiction, and advertising categories respectively. The 
context vector generator 82 0 generates context vectors for 
that set of sample texts resulting in a first summary 
context vector 808, a second summary context vector 810, and 
an nth summary context vector 812. In the example given, the 
summary context vectors 808, 810, and 812 correspond to the 
categories of news, fiction and advertising respectively. 
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The summary vectors are stored in a local data storao:- 
system . 

Referring to FIG. 8B, a sample of the source relaLc.J 
text 13 6 which is associated with the new program to re- 
classified is passed to the context vector generator i-.l: ■ 
which generates a program context vector 840 for »: 
program. The source related text 136 can be either re- 
source material 130, the EPG 140, or other text associate-,: 
with the source material. A comparison is made between 
actual program context vectors and the stored proo: win 
content context vectors by computing, in a dot pro:hi?c 
computation process 830, the dot product of the f. -„?..-! 
summary context vector 808 with the program context vc-rt:i 
84 0 to produce a first dot product 814. Similar opera: : :u 
are performed to produce second dot product 816 and nth .i." t 
product 818. 

The values contained in the dot products 814, 8IC .nz.i 
818, while not probabilistic in nature, can be expressed :r; 
probabilistic terms using a simple transformation in wh:c::i 
the result represents a confidence level of assigning 
corresponding content to that program. The trans i orm- • ;i 
values add up to one. The dot products can be used 
classify a program, or form a weighted sum •;: i 
classifications which results in the program characteri ci i rru 
vectors 150. In the example given, if the source rel.;*. 
text 136 was from an advertisement, the nth dot product, -. i -■ 
would have a high value, indicating that the advert i * 
category was the most appropriate category, and assigrun : t 
high probability value to that category. If the dot prod*; .* 
corresponding to the other categories were signif id:.* ; y 
higher than zero, those categories would be assignc-.i 
value, with the result being the program charactering i r.* 
vectors 150 as shown in FIG. 9D. 

For the sub-categories, probabilities obtained from t. hv 
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content pertaining to the same sub-category 146 are summed 
to form the probability for the new program being in that 
sub-category 146. At the sub-category level, the same method 
is applied to compute the probability of a program being 
5 from the given category 144. The three levels of the program 
classification system; the category 144 , the sub- category 
14 6 and the content, are used by the program 
characterization process 800 to form the program 
characteristics vectors 150* which are depicted in FIGS . 9D- 
10 9F . 

The program characteristics vectors 150 in general are 
represented in FIGS. 9A through 9f/fiG5. 9A, 9B and 9C are 
an example of deterministic program vectors. This set of 
vectors is generated when the program characteristics are 
15 well defined, as can occur when the source related text 13 6 
or the EPG 14 0 contains specific fields identifying the 
category 144 and the sub-category 146. A program rating can 
also provided by the EPG 14 0. 

In the case that these characteristics are not 
2 0 specified, a statistical set of vectors is generated from 
the process described in accordance with FIG . 8 . FIG . 9D 
shows the probability that a, program being watched is from 
the given category 144. The categories are listed in the X- 
axis. The sub-category 146 is also expressed in terms of 
25 probability. This is shown in FIG. 9E. The content component 
of this set of vectors is a third possible level of the 
program classification, and is illustrated in FIG. 9F. 

FIG. 10A illustrates sets of logical heuristics rules 
which form part of the heuristic rules 160. In a preferred 
embodiment, logical heuristic rules are obtained from 
sociological or psychological studies. Two types of rules 
are illustrated in FIG. 10A. The first type links an 
individual ' s viewing characteristics to demographic 
characteristics such as gender , age, and income level . A 
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channel changing rate rule 1030 attempts to determine gender 
based on channel change rate. An income related channel 
change rate rule 1010 attempts to link channel change rates 
to income brackets. A second type of rules links particular 
5 programs to particular audience, as illustrated by a gender 
determining rule 1050 which links the program category 
144 /sub-category 14 6 with a gender. The result of the 
application of the logical heuristic rules illustrated in 
FIG. 10A are probabilistic determinations of factors 

10 including gender, age, and income level. Although a specific 
set of logical heuristic rules has been used as an example, 
a wide number of types of logical heuristic rules can be 
used to realize the present invention. In addition, these 
rules can be changed based on learning within the system or 

15 based on external studies which provide more accurate rules. 

FIG. 10B illustrates a set of the heuristic rules 160 
expressed in terms of conditional probabilities. In the 
example shown in FIG. 10B, the category 144 has associated 
with it conditional probabilities for demographic factors 

20 such as age, income, family size and gender composition. 
The category 144 has associated with it conditional 
probabilities that represent probability that the viewing 
group is within a certain age group dependent on the 
probability that they are viewing a program in that category 

25 144. 

FIG. 11 illustrates an entity- relationship diagram for 
the generation of the program demographic vectors 170. In a 
preferred embodiment, the heuristic rules 160 are applied 
along with the program characteristic vectors 150 in a 
30 program target analysis process 1100 to form the program 
demographic vectors 170. The program characteristic vectors 
150 indicate a particular aspect of a program, such as its 
violence level. The heuristic rules 160 indicate that a 
particular demographic group has a preference for that 
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program. As an example, ic may be the case that young male, 
have a higher preference for violent programs than cth-.: 
sectors of the population. Thus, a program which has Lh-- 
program characteristic vectors 150 indicating a hn:i. 
probability of having violent content, when combined wlz.:, 
the heuristic rules 160 indicating that, "young males 1 . h- 
violent programs," will result, through the program tar..-: 
analysis process 1100, in the program demographic vec-.or,- 
170 which indicate that there is a high probability thac t:,:- 
program is being watched by a young male. 

The program target analysis process 1100 can r- 
realized using software programmed in a Variety of langu.- i- .- 
which processes mathematically the heuristic rules 160 L 
derive the program demographic vectors 170. The t^l l 
representation of the heuristic rules 160 illustrated xr. 
FIG. 10B expresses the probability that the individual <..-■ 
household is from a specific demographic group based or. 
program with a particular category 144. This can t- 
expressed, using probability terms as follow »r!,., 
probability that the individuals are in a given demograr:nc 
group conditional to the program being in a given category" . 
Referring to FIG. 9D, the probability that the group ha:.-- 
certain demographic characteristics based on the pro-.; r.-.n. 
being in a specific category is illustrated. 

Expressing the probability that a program is destJiv_.J 
to a specific demographic group can be determined re- 
applying Bayes rule. This probability is the sum of t ri- 
conditional probabilities that the demographic group l:kv: 
the program, conditional to the category 144 weighted rv r. ::• 
probability that the program is from that category 144. : :i , ; 
preferred embodiment, the program target analysis 
calculate the program demographic vectors by application c f 
logical heuristic rules, as illustrated in FIG. 10A, an:i r/ 
application of heuristic rules expressed as conditio-... 1 
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probabilities as shown in FIG. 10B. Logical heuristic rules 
can be applied using logical programming and fuzzy logic 
using techniques well understood by those skilled in the 
art, and are discussed in the text by s. V. Kartalopoulos 
5 entitled ^Understanding Neural Networks and Fuzzy Logic" 
which is incorporated herein by reference. 

Conditional probabilities can be applied by simple 
mathematical operations multiplying program context vectors 
by matrices of conditional probabilities. By performing 

10 this process over all the demographic groups, the program 
target analysis process 1100 can measure how likely a 
program is to be of interest to each demographic group. 
Those probabilities values form the program demographic 
vector 170 represented in FIG. 12. 

15 As an example, the heuristic rules expressed as 

conditional probabilities shown in FIG. 10B are used as part 
of a matrix multiplication in which the program 
characteristics vector 150 of dimension N # such as those 
shown in FIGS. 9A-9F is multiplied by an N x M matrix of 

20 heuristic rules expressed as conditional probabilities, such 
as that shown in FIG. 10B. The resulting vector of 

dimension M is a weighted average of the conditional 
probabilities for each category and represents the household 
demographic characteristics 190. Similar processing can be 

25 performed at the sub- category and content levels. 

FIG. 12 illustrates an example of the program 
demographic vector 170, and shows the extent to which a 
particular program is destined to a particular audience. 
This is measured in terms of probability as depicted in FIG. 

30 12. The Y-axis is the probability of appealing to the 
demographic group identified on the X-axis. 

FIG. 13 illustrates an entity-relationship diagram for 
the generation of household session demographic data 1310 
and household session interest profile 1320. In a preferred 
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embodiment, the subscrxber selection data no xs used along 
wxth the program characteristics vectors 150 in a sessxon 
characterization process 1300 to generate the household 
sessxon interest profile 1320. The subscriber selection data 
• HO indicates what the subscrxber is watching, for how lone 
and at what volume they are watching the program. 

in a preferred embodiment, the session characterization 
process 1300 forms a weighted average of the program 
characteristics vectors 150 in which the time duration the 
program is watched is normalized to the session time 
(typically defined as the time from which the unit was 
turned on to the present). The program characteristics 
vectors 150 are multiplied by the normalized time duration 
(which is less than one unless only one program has been 
vxewed) and summed with the P revxous value. Time duration 
data, along with other subscriber viewing information, is 
available from the subscriber selection data no. The 
resulting weighted average of program characteristics 
vectors forms the household session interest profile 1320, 
with each program contributing to the household session 
interest profile 1320 according to how long it was watched. 
The household session interest profile 1320 is normalized to 
produce probabilistic values of the household programming 
interests during that session. 

In an alternate embodiment, the heuristic rules 160 are 
applied to both the subscriber selection data no and the 
program characteristics vectors 150 to generate the 
household session demographic data 1310 and the household 
session interest profile 1320. In this embodiment, weighted 
averages of the program characteristics vectors 150 are 
formed based on the subscriber selection data 110, and the 
heuristic rules 160 are applied. m the case of logical 

heuristic rules as shown in FIG inz\ i^-s^ = n 

WJ xn I OA, logical programming 

can be applied to make determinations regarding the 
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household session demographic data 1310 and the household 
session interest profile 1320. In the case of heuristic 
rules in the form of conditional probabilities such as those 
illustrated in FIG. 10B, a dot product of the time averaged 
5 values of the program characteristics vectors can be taken 
with the appropriate matrix of heuristic rules to generate 
both the household session demographic data 1310 and the 
household session interest profile 1320. 

Volume control measurements which form part of the 

1C subscriber selection data 110 can also be applied in the 
session characterization process 1300 to form a household 
session interest profile 1320. This can be accomplished by 
using normalised volume measurements in a weighted average 
manner similar to how time duration is used. Thus, muting a 

15 show results in a zero value for volume, and the program 
characteristics vector 150 for this show will not be 
averaged into the household session interest profile 1320. 

FIG. 14 illustrates an entity-relationship diagram for 
the generation of average household demographic 

20 characteristics and session household demographic 
characteristics 190. A household demographic 

characterization process 1400 generates the household 
demographic characteristics 190 represented in table format 
in FIG. 15. The household demographic characterization 

25 process 1400 uses the household viewing habits 195 in 
combination with the heuristic rules 160 to determine 
demographic data. For example, a household with a number of 
minutes watched of zero during the day may indicate a 
household with two working adults. Both logical heuristic 

30 rules as well as rules based on conditional probabilities 
can be applied co the household viewing habits 195 to obtain 
the household demographics characteristics 190. 

The household viewing habits 195 is also used by the 
system to detect out -of -habits events. For example, if a 
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household with a zero value for the minutes watched column 
702 at late night presents a session value at that time via 
the household session demographic data 1310, this session 
will be characterized as an out-of -habits event and . the 
system can exclude such data from the average if it is 
highly probable that the demographics for that session are 
greatly different than the average demographics for the 
household. Nevertheless, the results of the application of 
the household demographic characterization process 1400 to 
the household session demographic data 1310 can result in 
valuable session demographic data, even if such data is not 
added to the average demographic 'characterization of the 
household. 

FIG. 15 illustrates the average and session household 
15 demographic characteristics. A household demographic 
parameters column 1501 is followed by an average value 
column 1505, a session value column 1503, and an update 
column 1507. The average value column 1505 and the session 
value column 1503 are derived from the household demographic 
characterization process 14 00. The deterministic parameters 
such as address and telephone numbers can be obtained from 
an outside source or can be loaded into the system by the 
subscriber or a network operator at the time of 
installation. Updating of deterministic values is prevented 
by indicating that these values should not be updated in the 
update column 1507. 

FIG. 16 illustrates an entity-relationship diagram for 
the generation of the household interest profile 180 in a 
household interest profile generation process 16C0. In a 
30 preferred embodiment, the household interest profile 
generation process comprises averaging the household session 
interest profile 1320 over multiple sessions and applying 
the household viewing habits 195 in combination with the 
heuristic rules 160 to form the household interest profile 
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180 which takes into account both the viewing preferences of 
the household as well as assumptions about 

households/subscribers with those viewing habits and program 
preferences . 

5 FIG. 17 illustrates the household interest profile 130 

which is composed of a programming types row 17 OS, a 
products types row 1707, and a household interests column 
1701, an average value column 1703, and a session value 
cclumn 1705. 

10 The product types row 1707 gives an indication as to 

what type of advertisement the household would be interested 
in watching, thus indicating what types of products could 
potentially be advertised with a high probability of the 
advertisement being watched in its entirety. The 

15 programming types row 1709 suggests what kind of programming 
the household is likely to be interested in watching. The 
household interests column 1701 specifies the types of 
programming and products which are statistically 
characterized for that household. 

20 As an example of the industrial applicability of the 

invention, a household will perform its normal viewing 
routine without being requested to answer specific questions 
regarding likes and dislikes. Children may watch television 
in the morning in the household, and may change channels 

25 during commercials, or not at all. The television may 
remain off during the working day, while the children are at 
school and day care, and be turned on again in the evening, 
at which time the parents may "surf" channels, mute the 
television during commercials, and ultimately watch one or 

30 two hours of broadcast programming. The present invention 
provides the ability to characterize the household, and may 
make the determination that there are children and adults in 
the household, with program and product interests indicated 
in the household interest profile 180 corresponding to a 
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family of chat composition. a household with two retired 
adults will have a completely different characterization 
which will be indicated in the household interest profile 
180 . 

Although the present invention has been largely 
described in the context of a single computing platform 
receiving programming, the SCS 100 can be realized as part 
of a client-server architecture, as illustrated in FIG. 13. 
Referring to FIG. 18, residence 1800 contains a personal 
computer (PC) 1820 as well as the combination of a 
television 1810 and a set-top 1808, which can request and 
receive programming. The equipment in residence 1800, or 
similar equipment in a small or large business environment, 
forms the client side of the network as defined herein. 
Programming is delivered over an access network 1830, which 
may be a cable television network, telephone type network, 
or other access network. Information requests are made by 
the client side to a server 1840 which forms the server side 
of the network. Server 184 0 has content locally which it 
provides to the subscriber, or requests content on behalf of 
the subscriber from a third party content provider 1860, as 
illustrated in FIG. 18. Requests made on behalf of the 
client side by server 1840 are made across a wide area 
network 1850 which can be the Internet or other public or 
private network. Techniques for making requests on behalf of 
a client are frequently referred to a proxy techniques and 
are well known to those skilled in the art. The server side 
receives the requested programming which is displayed on PC 
1820 or television 1810 according to which device made the 
request . 

In a preferred embodiment the server 134 0 maintains the 
subscriber selection data 110 which it is able to compile 
based on its operation as a proxy for the client side. 
Retrieval of source related information and the program 
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target analysis process 1100, the program characterization 
process 800, the program target analysis process 1100, the 
session characterization process 1300, the household 
demographic characterization process 1400, and the household 
interest profile generation process 1600 can be performed by 
server 184 0. 

Referring to FIG. 19 an advertisement monitoring table 
is illustrated, in which an advertisement ID (AD ID) column 
1915 contains a numerical Id for an advertisement which was 
transmitted with the advertisement in the form of a Progiram 
ID, http address, or other identifier- which is uniquely 
associated with the advertisement. A product column 1921 
contains a product description which indicates the type of 
product that was advertised. A brand column 1927 indicates 
the brand name of the product or can alternatively list a 
generic name for that product. A percent watched column 1933 
indicates the percentage of the advertisement the subscriber 
viewed. In an alternate embodiment, a letter rating or 
other type of rating is used to indicate the probability 
that the advertisement was watched. A volume column 1937 
indicates the volume level at which the advertisement was 
watched . 

As an example of the industrial applicability of the 
invention, a manufacturer may develop an advertising 
strategy which includes the insertion of advertisements 
during popular evening programs. The costs for such ad 
insertions can be extremely high. In order to insure the 
cost effectiveness of this advertising strategy, the 
manufacturer has the advertisements placed during less 
watched but similar programs and monitors how subscribers 
react, and can determine approximately how many times the 
advertisement has been watched out of all of the possible 
viewings . This data can be used to confirm the potential 
effectiveness of the advertisement and to subsequently 
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determine if purchasing the more expensive time during 
evening programming will be cost-effective, or if the 
advertisement should be modified or placed in ether 
programming . 

Continuing this example, the manufacturer may place an 
advertisement for viewing during "prime time" for an initial 
period but can subsequently cancel broadcasts of the 
advertisement if it is found that the majority of 
subscribers never see the advertisement. 

Although this invention has been illustrated by 
reference to specific embodiments, it will be apparent to 
those skilled in the art that Various changes and 
modifications may be made which clearly fall within the 
scope of the invention. The invention is intended to be 
protected broadly within the spirit and scope of the 
appended claims . 
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Claims 

What is claimed is: 

1. A data processing system for monitoring advert isener.t. : : 
watched by a subscriber, said data processing systev 
comprising : 

(a) computer processor means for processing data; 

(b) storage means for storing data on a storage 
medium; 

(c) first means for monitoring subscriber 
activity wherein said first means includes 
recording means for storing subscriber 
selections; 

(d) second means for retrieving advertisement 
related information wherein said advertisement 
related information contains descriptive fields 
corresponding to said advertisement; 

(e) third means for processing information 
wherein said third means includes means for 
determining the extent to which an advertisement 
is viewed by said subscriber; and 

(f) fourth means for storing said descriptive 
fields and said determination of the extent to 
which said advertisement is viewed by said 
subscriber . 

2. The system described in claim 1 wherein said first mv.in: : 
for monitoring subscriber activity further comprises mv i:;.- 
for monitoring volume levels wherein said volume lev*. lr 
correspond to subscriber selection volume levels. 

3. The system described in claim 1 further comprising: 

(g) fifth means for determining a subscriber 
product interests profile; and 
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(h) sixth means for storing said subscriber 
products interests profile. 

4. The system described in claim 1 wherein said second 
means for retrieving advertisement related information 
further comprises a means for context mining of textual 
information associated with said selected source material. 

5. The system described in claim 4 wherein said textual 
information is text derived from closed- captioning data 
associated with said advertisement. 

6. The system described in claim 5 wherein said text 
derived from closed-captioning data associated with said 
advertisement includes a product name field. 

7. The system described in claim 4 wherein said text 
derived from closed-captioning data associated with said 
advertisement includes a product brand field. 

8. A client-server based data processing system for 
monitoring advertisements watched by a subscriber, said 
client-server based data processing system comprising: 

(a) first computer processor means at a client 
side for receiving and displaying advertisements 
wherein said first computer means is capable of 
transmitting channel change requests; 

(b) second computer processor means at a server 
side for receiving said channel change requests 
and for processing data; 

(c) second storage means associated with second 
computer processor means for storing data on a 
storage medium; 
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(d) first means at said server side for 
monitoring subscriber activity wherein said 
first means for monitoring subscriber activity 
includes receiving means for receiving 
subscriber channel change requests, recording 
means for storing subscriber channel change 
requests ; 

(e) second means at said server side for 
retrieving advertisement related information 
wherein said advertisement related information 
contains descriptive fields corresponding to an 
advertisement; 

(f) third means at said server side for 
processing information wherein said third means 
includes means for determining the extent to 
which an advertisement is viewed by said 
subscriber; and 

(g) fourth means at said server side for storing 
said descriptive fields and said determination 
of the extent to which said advertisement is 
viewed by said subscriber. 


9. The system described in claim 8 further comprising: 

(h) fifth means for determining a subscriber 
product interests profile; and 

(i) sixth means for storing said subscriber 
product interests profile. 


10 . The system described in claim 8 wherein said second 
means for retrieving advertisement related information 
further comprises a means for context mining of textual 
information associated with said selected source material. 
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11. The system described in claim 10 wherein said textual 
information is text derived from closed-captioning date, 
associated with said advertisement. 

12. The system described in claim 11 wherein said tex: 
derived from closed-captioning data associated with saic: 
advertisement includes a product name field. 

13. The system described in claim 11 wherein said tex: 
derived from closed-captioning data associated with said 
advertisement includes a product brand field. 

14. A data processing system for generating a subscriber 
profile vector, said data processing system comprising: 

(a) computer processor means for processing data; 

(b) storage means for storing data on a storage 
medium; 

(c) first means for monitoring subscriber 
activity wherein said first means includes 
recording means for storing subscriber selection 
data wherein said subscriber selection data 
corresponds to selected source material; 

(d) second means for retrieving source related 
information wherein said source related 
information contains descriptive fields 
corresponding to said selected source material; 

(e) third means for processing information 
wherein said third means includes means for 
processing said subscriber selection data with 
respect to said descriptive fields to form said 
subscriber profile vector; and 

(f) fourth means for storing said subscriber 
profile vector. 
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15. The system described in claim 14 wherein said f irst 
means for monitoring subscriber activity further comprises 
means for monitoring time durations wherein said time 
durations correspond to viewing times of said selected 
source material . 

16. The system described in claim 14 wherein said first 
means for monitoring subscriber activity further comprises 
means for monitoring volume levels wherein said volume 
levels correspond to subscriber selection volume levels. 

17. The system described in claim 14 wherein said 
subscriber profile vector contains household demographic 
data indicating probabilistic measurements of household 
demographics . 

18 . The system described in claim 14 wherein said 

subscriber profile vector contains household program 
preference information indicating probabilistic measurements 
of household program interests. 

19. The system described in claim 14 wherein said 
subscriber profile vector contains household product 
preference information indicating probabilistic measurements 
of household product interests. 

20. The system described in claim 14 wherein said second 
means for retrieving source related information further 
comprises a means for context mining of textual information 
associated with said selected source material . 

21. The system described in claim 20 wherein said textual 
information is text derived from closed-cape ioning data 
associated with said selected source material . 
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22. The system described in claim 14 wherein said second 
means for retrieving source related information further 
comprises a means for retrieving information associated with 
said selected source material from an electronic program 
guide. 

23. The system described in claim 14 wherein said third 
means for processing information processes information ove 
a viewing session and wherein said subscriber profile vector 
corresponds to said viewing session. 


r 


24. The system described in claim 14 wherein said third 
means for processing information processes information over 
multiple viewing sessions and wherein said subscriber 
profile vector corresponds to an average value over said 
multiple viewing sessions. 

25. A data processing system for generating a subscriber 
profile vector, said data processing system comprising: 

(a) computer processor means for processing data; 

(b) storage means for storing data on a storage 
medium; 

(c) first means for monitoring subscriber 
activity wherein said first means includes 
recording means for storing subscriber selection 
data wherein said subscriber selection data 
corresponds to selected source material; 

(d) second means for retrieving source related 
information wherein said source related 
information contains descriptive fields 
corresponding to said selected source material; 
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(e) third means for generating a program 
characteristics vector based on said source 
related information; 

(f) fourth means for storing a set of heuristic 
rules ; 

(g) fifth means for processing information 
wherein said fifth means includes means for 
processing said subscriber selection data with 
respect to said program characteristics vector 
and said set of heuristic rules to form said 
subscriber profile vector; and 

(h) sixth means for storing said subscriber 
profile vector. 

26. The system described in claim 25 wherein said first 
means for monitoring subscriber activity further comprises 
means for monitoring time durations wherein said time 
durations correspond to viewing times of said selected 
source material . 

27. The system described in claim 25 wherein said first 
means for monitoring subscriber activity further comprises 
means for monitoring volume levels wherein said volume 
levels correspond to subscriber selection volume levels. 

28. The system described in claim 25 wherein said 
subscriber profile vector contains household demographic 
data indicating probabilistic measurements of household 
demographics . 

29. The system described in claim 25 wherein said 
subscriber profile vector contains a household session 
interest profile indicating probabilistic measurements of 
household interests . 
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30. A data processing system for generating a household 

demographic characteristics vector said rf a , = 

vc^uui, saia data processing 

system comprising: 

(a) computer processor means for processing data; 

(b) storage means for storing data on a storage 
medium; 

(c) first means for monitoring subscriber 
activity wherein said first means includes 
recording means for storing subscriber selects 
data wherein said subscriber selection dat< 
corresponds to selected source material ; 

(d) second means for generating household v le w ln g 
habits information wherein said household 
viewing habits information is generated from 
said subscriber selection data; 

(e) third means for storing a set of heuristic 
rules; 

(f) fourth means for processing information 
wherein said fourth means includes means for 
processing said subscriber selection data with 
respect to said set of heuristic rules to form 
said household demographic characteristics 
vector; and 

(9) fifth means for storing said household 
demographic characteristics vector. 

31. The system described in claim 30 wherein said fourth 
means for processing information processes information over 
a viewing session and wherein said household demographic 
characteristics vector corresponds to said viewing session. 

32. The system described in claim 30 wherein said fourth 
means for processing information processes information over 
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a period of multiple viewing sessions wherein said household 
demographic characteristics vector corresponds to an average 
value over said multiple viewing sessions. 


33 . A data processing system for generating a subscriber 
profile vector in a client-server based architecture, said 
data processing system comprising: 

(a) first computer processor means at a client 
side for requesting and displaying source 
information wherein said first computer means 
transmits a request for source material and 
receives and displays said source material ; 

(b) second computer processor means at a server 
side for processing data; 

(c) second storage means associated with second 
computer processor means for storing data on a 
storage medium; 

(d) first means at said server side for 
monitoring subscriber activity wherein said 
first means for monitoring subscriber activity 
includes receiving means for receiving 
subscriber requests for said source material, 
recording means for storing subscriber selection 
data wherein said subscriber selection data 
corresponds to a record of requests for said 
source material; 

(e) second means at said server side for 
retrieving source related information wherein 
said source related information contains 
descriptive fields corresponding to said source 
material ; 

(f) third means at said server side for 
processing information wherein said third means 
includes means for processing said subscriber 
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selection data with respect to said descriptive 
fields to form said subscriber profile vector; 
and 

(g) fourth means at said server side for storing 
said subscriber profile vector. 

34. The system described in claim 33 ' wherein said first 
means for monitoring subscriber activity further comprises 
means for monitoring time durations wherein said time 
durations correspond to viewing times of said selected 
source material. 


35. The system described in claim 33 wherein said first 
means for monitoring subscriber activity further comprises 


means for monitoring volume levels wherein said vol 
levels correspond to subscriber selection volume levels. 


ume 


36. The system described in claim 33 wherein said 
subscriber profile vector contains household demographic 
data indicating probabilistic measurements of household 
demographics . 

37. The system described in claim 33 wherein said 
subscriber profile vector contains household program 
preference information indicating probabilistic measurements 
of household program interests. 

38. The system described in claim 33 wherein said 
subscriber profile vector contains household product 
preference information indicating probabilistic measurements 
of household product interests. 

39. The system described in claim 33 wherein said second 
means for retrieving source related information further 
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information 


40. The system described in claim 39 wherein said textual 
information is text derived from closed-capt ioning data 
associated with said selected source material. 

41. The system described in claim 33 wherein said second 
means for retrieving sovlrce related information further 
comprises a means for retrieving information associated with 
said selected source material from an electronic program 
guide . 

42. The system described in claim 33 wherein said third 
means for processing information processes information over 
a viewing session and wherein said subscriber profile vector 
corresponds to said viewing session. 

43. The system described in claim 33 wherein said third 
means for processing information processes information over 
multiple viewing sessions and wherein said subscriber 
profile vector corresponds to an average value over said 
multiple viewing sessions. 

44 . A data processing system for generating a subscriber 
profile vector in a client- server based architecture, said 
data processing system comprising: 

(a) first computer processor means at a client 
side for requesting and displaying source 
information wherein said first computer means 
transmits a request for source material and 
receives and displays said source material ; 

(b) second computer processor means at a server 
side for processing data; 


WO 00/33160 

PC77US99/28528 

(c) second storage means associated with second 
computer processor means for storing data on a 
storage medium ; 

(d) first means at said server side for 
monitoring subscriber activity wherein said 
first means for monitoring subscriber activity 
includes receiving means f or receiving 
subscriber requests for said source material, 
recording means for storing subscriber selection 
data wherein said subscriber selection data 
corresponds to a record of requests for said 
source material ; 

(e) second means at said server side for 
retrieving source related information wherein 
said source related information contains 
descriptive fields corresponding to said source 
material ; 

(f) third means at said server side for 
generating a program characteristics vector 
based on said source related information; 

(g) fourth means at said server side for storing 
a set of heuristic rules; 

(h) fifth means at said server side for 
processing information wherein said fifth means 
includes means for processing said subscriber 
selection data with respect to said program 
characteristics vector and said set of heuristic 
rules to form said subscriber profile vector; 
and 

(i) sixth means at said server side for storing 
said subscriber profile vector. 

45. The system described in claim 44 wherein said first 
means for monitoring subscriber activity further comprises 
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means for monitoring time durations wherein said t ! r:i-- 
durations correspond to viewing times of said selc./-r i 
source material . 

46. The system described in claim 44 wherein said i 
means for 'monitoring subscriber activity further compi i 
means for monitoring volume levels wherein said v:-.*_r:-* 
levels correspond to subscriber selection volume levels. 


47. The system described in claim 44 wherein 

subscriber profile vector contains household demogr.i 
data indicating probabilistic measurements of hous-.- 
demographics . 


i \ i 


48. The system described in claim 44 wherein r.ixa 
subscriber profile vector contains a household sck.-i . 
interest profile indicating probabilistic measureraep/.r c L 
household interests . 

49. A data processing system for generating a housei.'. i ;J 
demographic characteristics vector in a client-server b*...--*.^i 
architecture, said data processing system comprising: 

(a) first computer processor means at a clienr. 
side' for requesting and displaying source- 
information wherein said first computer meanr. 
transmits a request for source material an-i 
receives' and displays said source material ; 

(b) second computer processor means at a serve; 
side for processing data; 

(c) first means at said server side for 
monitoring subscriber activity wherein said 
first means includes recording means for storing 
subscriber selection data wherein said 
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subscriber selection data corresponds to 
selected source material; 

(d) second means at said client side for 
generating household viewing habits information 
wherein said household viewing habits 
information is generated from said subscriber 
selection data; 

(e) third means at said server side for storing a 
set of heuristic rules; 

(f) fourth means at said server side for 
processing information wherein said fourth means 
includes means for processing * said subscriber 
selection data with respect to said set of 
heuristic rules to form said household 
demographic characteristics vector; and 

(g) fifth means at said server side for storing 
said household demographic characteristics 
vector . 
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The system described in claim 49 wherein said fourth 
means for processing information processes information over 
a viewing session and wherein said household demographic 
characteristics vector corresponds to said viewing session. 

51. The system described in claim 49 wherein said fourth 
means for processing information processes information over 
a period of multiple viewing sessions wherein said household 
demographic characteristics vector corresponds to an average 
value over said multiple viewing sessions. 
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